Field of Invention
Broadly, the present invention relates to the field of camera calibration, and more specifically to the field of virtual camera calibration, where the latter consists of the human-eye/optical see-through (OST) head-mounted display (HMD) system, as a part of an augmented reality (AR) unit. More particularly, the present invention relates to calibration of the virtual views of OST HMDs. Specifically, the present invention relates to stereo calibration of the latter, and anchoring perceived virtual 3D objects to a real 3D environment.
Description of Related Art
Head-mounted displays have become increasingly popular for 3D visualization, particularly for replacing real environments with virtual substitutes. In parallel, the field of augmented reality (AR) has likewise been growing. In an AR system, real environments are enriched by superposition of virtual items, which may include virtual objects or text that appears to float by a real object to which it refers. Thus, there are two main aspects of AR technology: the capacity to enhance visual content and the potential to extend and enhance a user's interaction with the real world.
An AR unit may provide a view (or display or image) of visual content as “seen” (or captured) by a virtual camera. As it is known in the art, a virtual camera displays an image of a virtual object from a prescribed angle (or field of view, FOV) as it would be captured, or seen, by a real camera if a real camera were positioned at the prescribed angle and the virtual object were a real object. The resultant view includes not only the real, physical items in a real scene, but also virtual objects inserted into the real scene. In such an augmented reality (AR) unit, the virtual camera view may be provided by an optical see-through (OST) head-mounted display (HMD) system (i.e., an OST HMD system).
In a human-eye/optical see-through display, an individual image is designated for, and observable by, a select eye(s) of a human user. If the human-eye OST is a monoscopic display then only one image for one (or both) of a user's two eyes is provided. If the human eye OST is a stereoscopic display then two separate images, one for each of the user's two eyes, are provided. For example, a human eye OST may have one display directly over one eye of a human user, and optionally have another, separate display directly over the other eye of the human user. Alternatively, a human eye OST may project a separate image directly into separate eyes of the human user, or otherwise control separate images to be separately viewed by each of the one or two eyes of the human user.
Typically, this includes establishing a method of consistently translating between real-world coordinates of a real-world scene and virtual coordinates in a computer generated virtual scene corresponding to the real-word scene. In order for an AR system to provide an immersive experience, it is desirable that the virtual objects be rendered and placed within the virtual camera display in three dimensions (3D), i.e. a stereoscopic image, or display. This, in turn, requires capturing 3D information of the real-world scene. Such 3D information may be obtained by such means as “time-of-flight” sensors, visual tracking, inertial sensors, mechanically linked trackers, phase-difference sensors, and/or hybrid systems. Irrespective of the 3D capturing method used, an AR system requires calibration between the captured real-world information and the computer-generated, virtual world information. This requires calibrating the AR's virtual camera view to the real-world captured data.
There are many methods of calibrating a virtual camera to real-world captured information, but such methods are often time-consuming and/or have high computing resource requirements. It is desirable that an AR system be portable and wearable by a human user, which places limits on computer architecture, and thus limits on available computing resources. It is further desirable that an AR system provide a virtual camera display in real-time, which is hindered by the limitations on available computing resources.
What is needed is a system to simplify the calibrating of real-world captured data of real-world scenes/objects and a computer-generated virtual scene/object.
It is an object of the present invention to provide a system/method for simplifying the calibration of an AR system to a real-world scene.
It is a further object of the present invention to provide an AR system capable of rendering virtual camera displays combining real-world captured information with virtual object information in real-time.